Papers with fine-grained alignment
Boosting Textural NER with Synthetic Image and Instructive Alignment (2024.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a key task reliant on textual data. |
| Approach: | They propose a method to transform NER into a multimodal task by using images from the internet as auxiliaries. |
| Outcome: | The proposed method surpasses all text-only baselines and improves F1 score by 1.4% to 2.3% on prominent MNER datasets. |
A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation (2020.acl-main)
Copied to clipboard
Jan Deriu, Katsiaryna Mlynchyk, Philippe Schläpfer, Alvaro Rodrigo, Dirk von Grünigen, Nicolas Kaiser, Kurt Stockinger, Eneko Agirre, Mark Cieliebak
| Challenge: | Existing methods to efficiently construct corpus for question answering over structured data are time-consuming and cost-intensive. |
| Approach: | They propose a method to efficiently construct a corpus for question answering over structured data. |
| Outcome: | The proposed method triples the annotation speed while maintaining complexity of queries. |
CharacterCraft: Bridging the Literature-Reality Dialogue Gap for Practical Role-Playing Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dialogue datasets have a bias between query distributions and real-world user language usage. |
| Approach: | They propose a framework for Chinese role-playing and a robust evaluation method . they propose specialized Chinese dialogue extraction model and specialized memory retrieval module . |
| Outcome: | The proposed framework extracts character dialogue from novels and ensures high data quality. |
FineLAP: Taming Heterogeneous Supervision for Fine-grained Language-Audio Pretraining (2026.acl-long)
Copied to clipboard
| Challenge: | Existing audio-language models excel at clip-level understanding but struggle with frame-level tasks. |
| Approach: | They propose a novel training paradigm that advances both clip- and frame-level alignment in CLAP with heterogeneous data. |
| Outcome: | The proposed training paradigm improves both clip- and frame-level alignment in CLAP with heterogeneous data. |
HELFI: a Hebrew-Greek-Finnish Parallel Bible Corpus with Cross-Lingual Morpheme Alignment (2020.lrec-1)
Copied to clipboard
Anssi Yli-Jyrä, Josi Purhonen, Matti Liljeqvist, Arto Antturi, Pekka Nieminen, Kari M. Räntilä, Valtter Luoto
| Challenge: | Parallel editions of Bible translations have existed for 1,800 years, but there is a steadily growing interest to attach a fine-grained alignment to the numerous translations of the Bible and other parallel texts. |
| Approach: | They propose to produce an openly shareable, fine-grained alignment for parallel Bibles using only freely available text editions and annotations. |
| Outcome: | The proposed dataset contains the source texts and translations, morphological analyses and cross-lingual morpheme alignments. |
VQA-Augmented Machine Translation with Cross-Modal Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multimodal machine translation methods often extract visual features using pre-trained models while learning text features from scratch, leading to representation imbalance. |
| Approach: | They propose a cross-modal VQA-augmented multimodal machine translation method . it aligns image-source text pairs and image-question text pairs through dual-text contrastive learning . |
| Outcome: | The proposed method outperforms state-of-the-art methods on multiple evaluation metrics. |
Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work uses large language models (LLMs) to evaluate natural language process tasks, but there are shortcomings in current LLMs. |
| Approach: | They examine the alignment between LLM evaluators and human annotators by comparing conventional and alignment tasks with different evaluation criteria. |
| Outcome: | The proposed models excel in general criteria, such as fluency, but face challenges with complex criteria, including numerical reasoning. |
DaNet: Dual-Aware Enhanced Alignment Network for Multimodal Aspect-Based Sentiment Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods assume a direct alignment between images and aspects, matching the entire image with a corresponding aspect. Existing algorithms assume 'direct alignment' between images, introducing noise. |
| Approach: | They propose a Dual-Aware Enhanced Alignment Network (DaNet) that can enhance fine-grained multimodal aspect-image alignment and denoising. |
| Outcome: | The proposed system outperforms existing methods in three subtasks and is available on https://github.com/***/DaNet. |
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding (2023.acl-long)
Copied to clipboard
Haoli Bai, Zhiguang Liu, Xiaojun Meng, Li Wentao, Shuang Liu, Yifeng Luo, Nian Xie, Rongfu Zheng, Liangwei Wang, Lu Hou, Jiansheng Wei, Xin Jiang, Qun Liu
| Challenge: | Existing solutions for visual document understanding lack granularity of document textlines. |
| Approach: | They propose a supervised pre-training program to leverage structural knowledge nested in document textlines to achieve fine-grained alignment between visual regions and texts. |
| Outcome: | The proposed system performs better on various VDU tasks in English and Chinese. |
NAIST-SIC-Aligned: An Aligned English-Japanese Simultaneous Interpretation Corpus (2024.lrec-main)
Copied to clipboard
| Challenge: | Simultaneous interpretation data is a task where an utterance is translated in real-time. |
| Approach: | They propose to use an automatically-aligned parallel English-Japanese SI dataset to make it suitable for model training. |
| Outcome: | The proposed model improves translation quality and latency over baselines. |
PVTNL: Prompting Vision Transformers with Natural Language for Generalizable Person Re-identification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Domain generalization person re-identification (DG-ReID) aims to train models on source domains and generalize to unseen target domains. |
| Approach: | They propose a framework to generalize person re-identification using a vision-language model . body-part cues are used to segment images into semantically coherent regions . |
| Outcome: | The proposed framework can generalize to unseen domains and generalize semantics to people . it leverages the pre-trained vision-language model BLIP to extract aligned visual and textual embeddings. |
COVER: Context-Driven Over-Refusal Verification in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have become increasingly prevalent in the field of Natural Language Processing (NLP), achieving unprecedented performance across linguistic tasks. |
| Approach: | They propose a framework to quantify and analyze context-driven over-refusal . they find that over-fusals depend on the task, system prompts, model family, and the number of retrieved documents. |
| Outcome: | The proposed framework quantifyes and analyzes the concept of context-driven over-refusal on two public corpora. |
DMSD: Dual-Modal Semantic Disentanglement for Compositional Zero-Shot Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Compositional Zero-Shot Learning (CZSL) is a new research paradigm that learns sub-concepts from seen compositions and recognizes unseen novel combinations. |
| Approach: | They propose a Dual-Modal Semantic Disentanglement framework that integrates visual and textual information to achieve effective sub-concept disentangling. |
| Outcome: | The proposed framework achieves state-of-the-art performance on three benchmark datasets . it integrates a class-centroid bridge module to guide class centroids toward the textual space . |